Dense Subsets of Pseudorandom Sets ∗ [ Extended

A theorem of Green, Tao, and Ziegler can be stated (roughly) as follows: ifR is a pseudorandom set, andD is a dense subset of R, then D may be modeled by a set M that is dense in the entire domain such that D and M are indistinguishable. (The precise statement refers to“measures” or distributions rather than sets.) The proof of this theorem is very general, and it applies to notions of pseudorandomness and indistinguishability defined in terms of any family of distinguishers with some mild closure properties. The proof proceeds via iterative partitioning and an energy increment argument, in the spirit of the proof of the weak Szemerédi regularity lemma. The “reduction” involved in the proof has exponential complexity in the distinguishing probability. We present a new proof inspired by Nisan’s proof of Impagliazzo’s hardcore set theorem. The reduction in our proof has polynomial complexity in the distinguishing probability and provides a new characterization of the notion of “pseudoentropy” of a distribution. A proof similar to ours has also been independently discovered by Gowers [2]. We also follow the connection between the two theorems and obtain a new proof of Impagliazzo’s hardcore set theorem via iterative partitioning and energy increment. While our reduction has exponential complexity in some parameters, it has the advantage that the hardcore set is efficiently recognizable. ∗A full version of this paper is available on ECCC [9] and a short exposition of the main theorem is also available on ArXiv [10]. †Research supported by US-Israel Binational Science Foundation Grant 2006060. ‡Work partly done while visiting Princeton University and the IAS. Research supported by the National Science Foundation under grants CCF0515231 and CCF-0729137 and by the US-Israel Binational Science Foundation grant 2006060. §Work done during a visit to U.C. Berkeley, supported by the Miller Foundation for Basic Research in Science and a Guggenheim Fellowship. This materials is also based on work supported by US-Israel Binational Science Foundation under grant 2006060, and the Office of Naval Research under grant N00014-04-1-0478.